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SUMMARY  PAGE 


THE  PROBLEM 

To  investigate  the  predictability  status  of  various  sentence 
lists  for  use  in  evaluating  hearing  abilities  of  Navy  personnel. 


FINDINGS 

The  results  indicate  that  sentence  predictability  plays  an 
important  role  in  the  interpretation  of  the  intelligibility  scores  ob¬ 
tained  using  sentence  tests.  Moreover,  the  predictability  status 
of  sentences  was  shown  to  vary  so  that  low  predictability  sentences 
can  be  used  to  determine  the  message  itself,  while  high  predict¬ 
ability  sentences  can  be  used  to  determine  one’s  ability  to  predict 
messages. 


APPLICATION 

The  predictability  status  of  sentences  can  be  manipulated  to 
provide  additional  evaluation  of  various  hearing  problems  incurred 
by  Navy  personnel.  Also,  sentence  lists  with  various  predictabil¬ 
ity  levels  may  be  used  .as  an  additional  tool  for  studying  the  ability 
to  understand  messages  from  fragmentary  information  such  as  in 
the  case  of  message  sending  from  hyperbaric  or  underwater  en¬ 
vironments  which  create  distortions  in  the  speech. 
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ABSTRACT 


The  purpose  of  this  study  was  to  evaluate  the  predictability 
status  of  several  sentence  lists  for  use  in  hearing  tests  of  Navy 
personnel.  One  hundred  and  eighty  normal -hearing  subjects, 
divided  into  six  groups,  listened  to  three  sentence  lists  presented 
in  one  of  six  word-elimination  conditions.  The  sentence  lists  were 
scored  in  terms  of  the  number  of  words  correctly  predicted  out  of 
the  number  of  words  eliminated.  The  following  results  were  ob¬ 
tained:  (1)  It  was  necessary  to  remove  between  20  to  25  keywords 
for  Sentence  Lists  B  and  D,  respectively,  before  obtaining  a  sig¬ 
nificant  score  reduction.  These  lists  were  found  to  be  highly  pre¬ 
dictable  and  quite  similar  with  regard  to  their  response  to  key¬ 
word  elimination.  (2)  Considerably  lower  scores  were  obtained 
for  Sentence  List  C  than  those  obtained  for  Lists  B  and  D.  Fur¬ 
thermore,  increased  degradation  through  increased  key-word 
elimination  did  not  significantly  affect  the  predictability  scores . 

It  was  concluded  that  List  C  is  considerably  less  predictable  than 
Lists  B  and  D.  (3)  The  error  curve  for  a  Synthetic  Sentence  List 
was  found  to  be  extremely  low  and  relatively  flat,  suggesting  that 
this  group  of  sentences  has  a  negligible  predictability  value. 
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STUDIES  OF  VOICE  COMMUNICATION  IN  THE  NAVY: 
SENTENCE  PREDICTABILITY 


INTRODUCTION 

Sentence  lists  have  received  renewed 
attention  (Berger1;  Giolas2;  Harris, 
Haines  and  Meyers  5;  Jerger,  Speaks  and 
Trammell4 )  as  measures  of  speech  intel¬ 
ligibility.  It  has  been  argued  that  sen¬ 
tences  present  a  more  natural  listening 
task  than  do  the  often  employed  mono¬ 
syllabic  word  lists  (Hirsh,  Davis, 
Silverman,  Eldert  and  Benson5)  and 
take  advantage  of  crucial  parameters 
utilized  in  understanding  connected 
speech.  The  Central  Institute  for  the 
Deaf  (CID)  Sentence  Lists  (Silverman 
and  Hirsh5)  have  been  suggested  as  one 
possible  message  for  clinical  use.  This 
group  of  sentence  lists  was  developed  to 
be  representative  of  colloquial  speech 
and  a  close  relationship  has  been  dem¬ 
onstrated  between  scores  obtained  using 
these  sentence  lists  and  a  sample  of 
continuous  discourse  (Giolas2) .  Fur¬ 
thermore,  the  lists  are  readily  avail¬ 
able  in  written  form  (Davis  and  Silver- 
man7),  require  little  administration 
time,  are  purported  to  include  ten 
equivalent  lists,  and  are  easy  to  score. 
On  the  other  hand,  it  has  been  shown 
that  speech  intelligibility  scores  ob¬ 
tained  with  these  sentences  are  rather 
high  under  low  levels  of  frequency  dis¬ 
tortion  (Giolas2).  One  possible  ex¬ 
planation  of  these  results  is  that  the 
sentences  comprising  this  series  are 
highly  redundant.  That  is,  a  large  por¬ 
tion  of  the  sentence  can  be  predicted  on 
the  basis  of  hearing  only  a  small  por¬ 
tion.  It  is  believed  that  this  group  of 
sentences  may  not  comprise  a  suffi¬ 
ciently  sensitive  test  to  detect  subtle 


differences  in  speech  intelligibility 
which  may  exist  under  minimally  de¬ 
graded  conditions .  Therefore,  the  pur¬ 
pose  of  the  present  study  was  to  assess 
the  predictability  status  of  selected  CID 
Sentence  Lists,  as  well  as  selected 
lists  from  two  other  series  (Harris , 
Haines,  Kelsey  and  Clack5,  Jerger, 
et  al4).  Theoretically,  a  low  predict¬ 
ability  status  would  be  desirable  if 
sentence  intelligibility  scores  are  to  be 
more  dependent  on  speech  discrimina¬ 
tion  ability  than  on  the  influence  of  con¬ 
textual  clues. 


METHOD 

Speech  Samples.  Scores  obtained  on 
CID  Sentence  Lists  B  and  D,  Revised 
CID  Sentence  List  C  and  ten  third-order 
synthetic  sentences,  constructed  by 
Jerger,  et  al4  were  compared  (see  Ap¬ 
pendix  A  for  these  four  sentence  lists). 
CID  Sentence  List  B  was  selected  in 
order  to  look  at  the  possible  predict¬ 
ability  factor  found  with  this  list  in  a 
previous  study  (Giolas9).  CID  Sentence 
List  D  was  employed  so  that  some  ob¬ 
servations  could  be  made  concerning 
list  equivalency  in  this  series .  Revised 
CID  Sentence  List  C  consists  of  an  al¬ 
tered  version  of  the  original  CID  List  C 
in  which  the  sentences  were  modified  to 
provide  greater  homogeneity  of  sentence 
length,  while  maintaining  the  colloquial 
speech  criterion.  This  list  was  in¬ 
cluded  in  order  to  investigate  the  effect 
of  sentence  length  on  sentence-imbedded 
word  predictability.  The  synthetic  sen¬ 
tences  were  included  to  provide  a 
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method  for  assessing  the  validity  of  the 
procedure  employed  to  measure  word 
predictability  in  sentences.  A  valid 
procedure  would  yield  a  low  predict¬ 
ability  score  in  that  contextual  clues  in 
synthetic  sentences  are  minimal. 

Word  Predictability  Procedure.  For 
purposes  of  this  study,  word  predict¬ 
ability  is  defined  as  the  property  of  a 
sentence  which  permits  the  prediction 
of  a  missing  word(s).  A  modified  ver¬ 
sion  of  the  "cloze  test*  procedure"  was 
employed  to  measure  word  predictabil¬ 
ity  within  a  sentence  structure.  The 
procedure  typically  involves  replacing  a 
portion  of  a  written  message  by  a  blank 
space.  It  was  originally  developed  to 
measure  the  comprehensibility  of  a 
written  message  by  requesting  the  sub¬ 
ject  to  predict  the  missing  words  (Tay¬ 
lor™'). 

In  the  present  study,  an  auditory 
cloze  test  procedure  was  employed  to 
measure  the  predictability  status  of 
sentence  lists  typically  used  in  speech 
intelligibility  testing.  It  was  assumed 
that  a  group  of  sentences  containing 
highly  predictable,  sentence-imbedded 
words  would  yield  high  speech  intelli¬ 
gibility  scores  as  a  function  of  message 
prediction  rather  than  message  recep¬ 
tion. 

All  sentence  groups  were  recorded 
on  magnetic  tape  by  a  male  speaker 
using  an  Ampex  Model  300  Tape  Re¬ 
corder  and  a  high  quality  microphone. 
Each  sentence  list  was  subjected  to  six 


*cloze  test  —  The  term  “cloze”  was  coined  from  the 
psychological  concept  of  closure.  Thus,  an  auditory  “cloze” 
procedure  implies  that  certain  words  are  deleted  from  a 
spoken  message  and  listeners  are  requested  to  close  the  gaps 
by  filling  in  the  missing  words. 


conditions  of  word  elimination.  Selec¬ 
tion  of  words  to  be  eliminated  was  con¬ 
ducted  on  a  random  rather  than  a  peri¬ 
odic  basis  to  avoid  systematic  elimina¬ 
tion  of  content  or  functional  words. 

While  there  is  some  evidence  that  cloze 
test  scores  may  vary  with  the  elimina¬ 
tion  of  different  words  (Greene 11 ),  little 
has  been  established  as  to  the  nature 
and  extent  of  the  score  changes  associ¬ 
ated  with  different  procedures, for  word 
deletion.  A  random  procedure  was 
employed,  since  it  is  the  most  effective 
method  of  avoiding  any  systematic  bias . 
CID  Sentence  lists  B  and  D  and  Revised 
CID  Sentence  List  C  typically  are 
scored  in  terms  of  a  key-word  count 
(Davis  and  Silverman7/;  consequently, 
only  keywords  were  considered  for 
elimination.  Since  the  suggested  scor¬ 
ing  procedure  for  the  Synthetic  Sen¬ 
tences  involves  a  total  word  count,  all 
words  were  considered  for  elimination. 
Word  elimination  was  accomplished  by 
carefully  cutting  out  the  selected  words 
and  replacing  them  with  an  identical 
amount  of  leader  tape.  Six  word-elim¬ 
ination  conditions  were  established: 
Condition  I,  10  words  eliminated;  Con¬ 
dition  II,  15  words  eliminated;  Condi¬ 
tion  m,  20  words  eliminated;  Condition 
IV,  25  words  eliminated;  Condition  V, 

30  words  eliminated;  and  Condition  VI, 
25  words  eliminated.  Condition  VI  was 
identical  to  Condition  IV,  except  that  a 
different  group  of  words  was  elimin¬ 
ated.  This  condition  was  included  in 
order  to  obtain  some  indication  of  the  in¬ 
fluence  of  word  selection  on  the  scores 
obtained.  All  treated  speech  samples 
were  re-recorded  to  eliminate  splice 
noise  and  to  insure  precise  presentation. 

TEST  ADMINISTRATION:  The  tape  re¬ 
cordings  were  played  on  an  Ampex 
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Model-300  Tape  Recorder,  through  a 
high-fidelity  amplifier  to  thirty  matched 
TDH-39  earphones .  The  sentences 
were  presented  to  the  left  ear  at  a 
comfortable  loudness  level  in  a  rela¬ 
tively  quiet  setting.  One  hundred  and 
eighty  normal-hearing  male  listeners 
were  divided  into  six  groups  of  30  sub¬ 
jects  each.  Each  group  listened  to  all 
four  sentence  lists  under  one  of  the 
word  elimination  conditions.  Answer 
sheets  and  pencils  were  distributed  to 
the  subjects  who  were  seated  in  tablet 
arm-chairs .  They  were  requested  to 
write  the  missing  words  in  the  appro¬ 
priate  blanks  provided  in  the  answer 
sheet.  The  answer  sheet  included  the 
same  amount  of  blanks  for  each  sen¬ 
tence  as  words  in  the  sentence.  A 
series  of  x’s  were  inserted  in  the 
blanks  for  the  words  not  eliminated. 

The  specific  instructions  were  as  fol¬ 
lows: 

The  purpose  of  this  test  is  to  see  how  well  you 
can  tell  what  the  missing  words  are  in  a  group  of 
sentences.  In  other  words,  you  will  hear  a  group 
of  sentences  in  which  certain  words  have  been 
eliminated.  You  are  to  guess  what  the  missing 
words  are.  Each  sentence  will  be  preceded  by  its 
number.  Listen  to  the  complete  sentence  and 
then  write  in  the  missing  word  or  words  in  the 
blanks  provided  for  you  on  the  answer  sheet. 

The  blanks  containing  x’s  indicate  where  in  the 
sentence  you  actually  heard  a  word  and  the 
empty  blanks  indicate  the  missing  words  you  are 
to  guess.  You  must  fill  in  each  blank  with  a  word. 

Do  not  leave  any  blanks,  PRIOR  TO  PRESENTA¬ 
TION  OF  SYNTHETIC  SENTENCES:  The  last 
group  of  sentences  will  have  very  little  meaning. 

Continue  to  guess  at  the  missing  words. 

The  subjects  were  given  60  seconds  to 
complete  the  blanks  for  each  sentence, 
but  the  full  60  seconds  were  seldom 
needed.  The  sentences  were  scored  in 
terms  of  the  number  of  words  accur¬ 
ately  predicted.  Approximate  spellings 
were  accepted  as  correct.  Unfilled 
blanks  were  counted  as  incorrect. 


RESULTS  AND  DISCUSSION 

Differences  Between  Conditions .  Means 
and  standard  deviations  of  scores  ob¬ 
tained  for  the  four  sentence  lists  studied 
within  each  word-elimination  condition 
are  summarized  in  Table  I.  The  means 
recorded  represent  the  number  of 
words  correct  out  of  the  total  number  of 
words  eliminated.  As  indicated  in  the 
procedures,  the  number  of  words 
eliminated  increased  with  each  condi¬ 
tion  and,  consequently,  each  mean 
must  be  viewed  in  relation  to  its  own 
reference  point.  For  example,  the 
mean  of  6. 60  listed  for  Condition  I  rep¬ 
resents  6. 60  words  correctly  predicted 
out  of  ten  words  eliminated,  while  the 
mean  for  Condition  II  represents  7. 63 
words  correctly  predicted  out  of  15 
words  eliminated,  etc.  For  purpose 
of  comparison,  the  means  have  been 
converted  into  percentages  and  plotted 
in  Figure  1.  A  zero  word-elimination 
was  not  included,  for  it  was  assumed 
that  this  condition  would  yield  a  score 
of  approximately  1 00% . 

Z  values  for  differences  between 
proportions  were  computed  between 
Condition  I  and  all  subsequent  Condi¬ 
tions  for  each  list.  A  significant  Z  was 
obtained  between  Conditions  I— III,  and 
I-V  for  CID  Sentence  List  B  (.05)  and 
between  Conditions  I -HI,  and  I-V  for 
CID  Sentence  List  D.  This  suggests 
that  it  is  necessary  to  remove  between 
20-25  of  the  keywords  before  a  signifi¬ 
cant  score  reduction  takes  place.  It 
was  concluded  that  Lists  CID-B  and  D  - 
may  be  considered  quite  similar  with 
respect  to  the  effects  of  key-word  elim¬ 
ination  as  accomplished  in  this  study. 

On  the  other  hand,  Revised  CID  Sen¬ 
tence  List  C  is  affected  quite  differently 
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TABLE  I  -  Summary  of  Means  and  Standard  Deviation  for  Sentence 
List  Scores  Under  Word-Elimination  Conditions 


j  Word  Elimination 
Condition 

Sentence  Lists 

CID 

List  B 

CID 

List  D 

Revised  CID 
List  C 

Synthetic 

List 

I  (10  words) 

Mean 

6.60 

6.03 

2.53 

.466 

SD 

.129 

1.423 

.212 

.118 

n  (15  words 

Mean 

7.63 

7.36 

5.66 

.560 

SD 

1.94 

2.24 

1.84 

.185 

III  (20  words) 

Mean 

11.13 

6.90 

7.43 

1.900 

SD 

2.46 

2.54 

1.80 

.310 

IV  (25  words) 

Mean 

6.43 

8.70 

3.93 

1.70 

SD 

2.26 

2.24 

1.  79 

1.25 

V  (30  words) 

Mean 

5.86 

8.56 

4.66 

1.13 

SD 

2.31 

2.17 

2.14 

.047 

VI  (25  words) 

Mean 

6.26 

7.96 

3.03 

2.76 

SD 

1.85 

2.62 

1.30 

.289 

by  key-word  elimination.  As  can  be  D  in  that  minimal  degradation  (10  key- 
seen  in  Figure  1,  there  is  a  consider-  word  elimination)  and  increased  deg- 

ably  lower  score  for  the  minimal  key-  radation  produced  no  significant  change 

word  elimination  condition  than  for  CID  in  predictability  scores.  It  is  sug- 
Sentence  Lists  B  and  D.  Furthermore,  gested  that  the  "filler"  words  used  to 
increased  degradation  through  in-  equate  the  length  of  the  sentences  corn- 

creased  key-word  elimination  did  not  prising  the  list  were  more  difficult  to 

significantly  affect  the  predictability  predict  and,  consequently,  reduced  the 

scores  for  subsequent  conditions.  No  list’s  predictability  status, 

significant  Z  values  were  obtained  be¬ 
tween  Condition  I  and  all  other  Condi-  The  error  curve  plotted  in  Figure  1 

tions  for  Revised  CED  Sentence  List  C.  for  the  Synthetic  Sentence  List  (SSL)  is 
It  was  concluded  that  Revised  CID  Sen-  relatively  flat.  While  a  significant  Z 
tence  List  C  is  considerably  less  pre-  (.  01)  was  obtained  between  Conditions  I- 

dictable  than  CID  Sentence  Lists  B  and  III  and  I-IV,  the  numerical  difference  is 
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PERCENT  of  WORDS  CORRECTLY  PREDICTED 


I  H  HI  H  TT  3ZT 


WORD-  ELIMINATION  CONDITIONS 

Fig.  1.  Comparison  of  Four  Sentence  lists  (CID  Lists— B 
and  D ,  Revised  CID  List—C ,  and  a  third  order 
Synthetic  Sentence  List)  in  terms  of  Percent  of 
Key  Words  Correct  for  Six  Word— Elimination 
Conditions. 

quite  small  and  inconsequential.  It  was 
concluded  that  the  SSL  group  has  a  neg¬ 
ligible  predictability  value. ,  A  low, 
relatively  flat  curve  for  the  SSL  group 
was  expected  in  that  the  anticipated 
predictability  value  for  non-meaningful 
sentences  is  quite  low.  In  a  sense,  the 
extremely  low  scores  obtained  for  the 
SSL  group  for  all  conditions  lend  sup¬ 
port  to  the  use  of  word  elimination  pro¬ 
cedures  to  study  word  predictability. 

Theoretical  and  Obtained  Scores:  The 
predictability  status  of  the  four  sen¬ 
tence  lists  investigated  in  this  study  is 
also  demonstrated  in  Figure  2 .  Figure 
2  graphically  illustrates  the  relation¬ 
ship  between  theoretical  scores  and  the 
scores  actually  obtained  for  each  word- 
elimination  condition.  The  theoretical 
curve  represents  points  at  which  speech 
intelligibility  scores  would  fall  if  the 
predictability  status  of  each  sentence 


list  studied  were  zero;  it  is  based  on  the 
assumption  that  all  eliminated  words 
would  not  be  identified  correctly.  The 
points  plotted  on  the  obtained  curve 
were  derived  by  combining  the  theoret¬ 
ical  score  and  the  number  of  words 
correctly  predicted  in  each  condition. 
For  example,  10  of  the  fifty  keywords 
were  eliminated  in  Condition  I.  If  none 
of  the  eliminated  words  was  predicted, 
the  theoretical  score  would  be  40. 
However,  the  actual  mean  number  of 
eliminated  words  correctly  predicted 
for  Condition  I  was  6.  60,  yielding  an 
obtained  score  of  46. 60  [theoretical 
score  (40)  and  number  of  words  pre¬ 
dicted  (6.60)].  The  curves  plotted  in 
Figure  2  include  the  theoretical  and  ob¬ 
tained  scores  for  each  condition.  Con¬ 
fidence  intervals  computed  (.01  levels) 
for  the  mean  number  of  words  correctly 
predicted  at  each  condition  were  ex¬ 
tremely  small  (see  Table  II).  These 
findings  indicate  there  is  a  real  separa¬ 
tion  between  the  theoretical  and  ob¬ 
tained  curves . 


Fig.  2.  Comparison  of  Theoretical  and  Obtained  Scores  for 
Four  Sentence  Lists  (CID  Lists— B  and  D,  also 
Revised  CID  List—C,  and  a  third  order  Synthetic 
Sentence  List)  under  Six  Word— Elimination 
Conditions. 
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TABLE  n  -  Means  and  .  01  Confidence  Intervals  For  All 
Sentence  Lists  at  Each  Condition 


Word  Elimination 
Condition 

Sentence  Lists 

CID 
List  B 

CID 
List  D 

R-CID 
List  C 

SSL 

I  (10  Words) 

Mean 

Confidence  Interval 

6.  60 
.633 

6.03 

.712 

2.53 

.105 

.466 

.057 

n  (15  Words) 

Mean 

Confidence  Interval 

7.63 

.973 

7.36 

1.12 

5.66 

.924 

.560 

.090 

m  (20  Words) 

Mean 

Confidence  Interval 

11.13 

1.23 

6.90 

1.28 

7.43 

.904 

1.900 

.154 

IV  (25  Words) 

Mean 

Confidence  Interval 

6.43 

1.135 

8.70 

1.12 

3.93 

.899 

1.70 

.627 

V  (30  Words) 

Mean 

Confidence  Interval 

5.86 

1.16 

8.56 

1.09 

4.66 

1,075 

1.13 

.129 

VI  (25  Words) 

Mean 

Confidence  Interval 

6.26 

.930 

7.96 

1.31 

3.03 

.651 

2.76 

.137 

Differences  Between  Lists:  Meannum-  not  heard.  The  mean  number  of  words 
ber  of  words  correctly  predicted  over  correctly  predicted  overall  conditions 
all  word  elimination  conditions  along  for  Revised  CID  Sentence  List  C  was 
with  the  _t-values  computed  between  4. 846  or  approximately  10%.  This  is  a 

these  means' are  listed  in  Table  HI.  lower  mean  (.01)  than  the  means  ob- 

CID  Sentence  Lists  B  and  D  yielded  ex-  tained  for  CID  Sentence  Lists  B  and 
tremely  similar  overall  means  which  D,  but  still  represents  an  appreciable 
support  the  earlier  conclusion  that  inflation  of  an  intelligibility  score  re- 

these  two  lists  are  quite  similar  with  suiting  from  contextual  cues.  The 
respect  to  sentence-imbedded  word  overall  mean  of  1. 153  obtained  for  the 

predictability.  Sentence  intelligibility  the  SSL  group  further  emphasizes  that 

scores  typically  obtained  with  either  of  this  group  of  sentences  has  the  lowest 
these  lists  will  be  inflated  by  approxi-  predictability  status  of  all  lists 
mately  eight  key  words  or  16%,  because  studied.  Furthermore,  intelligibility 
of  a  listener’s  ability  to  predict  words  scores  obtained  with  these  sentences 
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TABLE  m  -  Differences  in  Mean  Scores  Between  Test  Lists 

Over  all  Conditions 


Sentence 

Lists 

Means 

Differences 

Standard  Error 
of  Differences 

t 

CID-B 

7. 533 

CID-D 

7.513 

.020 

.274 

.072 

CID-B 

7.533 

R-CID-C 

4.846 

2.687 

.184 

♦14. 744 

CID-B 

7.533 

SSL 

1.153 

6.380 

.219 

♦29.132 

CID-D 

7.513 

R-CID-C 

4.846 

2.667 

.234 

♦11.397 

CID-D 

7.513 

SSL 

1.153 

6.360 

.192 

*33.125 

R-CID-C 

4.846 

SSL 

1.153 

3.693 

.187 

*19.748 

♦Significant  at  the  .01  level  of  confidence 


would  be  least  contaminated  by  sen¬ 
tence-imbedded  word  predictability. 

Word  Selection:  In  order  to  assess  the 
influence  of  specific  words  eliminated 
on  the  predictability  scores  obtained,  a 
comparison  was  made  between  Condi¬ 
tions  IV  and  VI.  As  you  will  recall, 
Condition  VI  was  identical  to  Condition 
IV,  except  that  a  different  group  of 
words  was  eliminated.  As  can  be  seen 
in  Table  HI,  the  means  for  these  two 
conditions  for  all  sentence  lists  are  ex¬ 
tremely  similar  and  Z  values  obtained 


for  the  differences  between  these  means 
were  non-significant  (.01). 

It  was  concluded  that  specific  words 
selected  for  elimination  did  not  have  an 
appreciable  effect  on  the  predictability 
of  the  sentence  lists  studied. 


CONCLUSIONS 

The  results  of  this  study  suggest  that 
sentence  predictability  plays  an  impor¬ 
tant  role  in  the  intelligibility  score  ob- 
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